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Executive Summary 

The Office of Shared Accountability (OSA) in Montgomery County Public Schools (MCPS) is 
conducting a multiyear evaluation of the Achieving Collegiate Excellence and Success (ACES) 
program. The ACES program is a collaboration between MCPS, Montgomery College (MC), 
and the Universities at Shady Grove (USG) to create a seamless pathway from high school to 
college completion. The first year of the program was implemented in both Grades 11 and 12 in 
10 schools. The schools were selected because they have a significant number of students who 
are underrepresented in higher education and who would be the first in their family to attend 
college. 

This is the first of a series of reports provided for the first year of the ACES program and focuses 
on a portion of the formative evaluation. This report examined two questions: 

• How is the ACES program perceived by students and program staff? 

• To what extent did the ACES program increase student knowledge and assist in their 
participation of the college application and financial aid process? 

Summary of Methodology 

A combination of methods was used to address the formative evaluation including surveys, 
interviews, and program documents. A total of 474 ACES students (a 49% response rate) 
completed an anonymous online survey about their participation in the first year of the program. 
All 10 ACES coaches and all 10 resource counselors (RCs) from the participating schools 
(a 100% response rate) completed an online survey at the end of the school year. These same 
staff members also were interviewed in the middle of the school year. 

Summary of Findings 

How is the ACES program perceived by students and program staff? 

Student surveys. Students learned about the program in a variety of ways including, but 
not limited to, their school counselor, during a class presentation, through the ACES coach, or 
during an assembly. 

Students had very positive ratings about their experiences in the program, with the vast majority 
of students agreeing with the statements addressing the different components of ACES. They 
were particularly positive about recommending the program to other students, the helpfulness of 
the information provided and guidance from the coach, and the coach’s availability as needed. 

Student respondents also gave positive ratings for their overall experience and the overall quality 
of the program, with only 8% and 6% giving it an average or poor rating, respectively. The vast 
majority of student respondents reported that the ACES coach was what they liked most about 
the program, and almost all of the comments submitted at the end of the survey either were 
positive about the ACES program or were personal thanks for the program or coaches. 
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Despite a high rating for the coach being available as needed, one fourth of the students reported 
that it was hard to find time to meet with the ACES coach, and one half reported that workshops 
were at times they could not attend. A variety of suggestions on improving the ACES program 
were made by one third of the student respondents including: more field trips (especially to 
colleges) and more flexible and accommodating schedule times for ACES activities. 
Additionally, one student mentioned a misconception among some of their peers that they only 
would be able to attend MC if they were a program participant. This coincides with a survey 
finding among a few of the counseling staff who disagreed that the ACES program helps 
students get accepted to a 4-year college. 

ACES coach and resource counselor surveys. Various methods for recruiting students 
were reported by coaches and RCs, which coincide with the student findings. These included: 
meeting with potential students, counselors referring students to coaches, presenting the ACES 
program to students in class or during an assembly, school advertising, and other methods. 
Although students were accepted into the program by meeting one of the stated criteria, 
interviews revealed some varying strategies for recruiting and enrolling students such as 
including everyone in the Advancement Via Individual Determination (AVID) program, 
recruiting English for Speakers of Other Languages students, and recruiting students receiving 
Free and Reduced-price Meal System services, while eliminating students with attendance issues 
or those not on track to graduate. 

Almost all coaches and RCs reported that coaches provided updates and information to the RC, if 
not other counselors, on ACES activities and ACES students. Almost all coaches reported in the 
survey that they personally attended counseling department meetings regularly or sometimes. 
The reasons given by coaches for not attending meetings were that they had to be at an MC 
meeting; the information at the counseling meeting was not related to ACES, or they were not 
included in the meetings. Although almost all coaches reported in the survey that they received 
student data through the Online Administrative Student Information System, interviews earlier in 
the year revealed that some coaches did not have easy access to student data. Lack of access to 
student data also was mentioned in the survey as a challenge coach’s faced. 

Ratings varied among the coach and RC survey respondents regarding the successful 
implementation of various aspects of the program this first year. Most of the coaches and RCs 
reported the collaboration between them was successful. Although ratings varied, a few coaches 
reported a difficultly with implementing the recruiting process, the enrollment process, and 
collaboration with the college and career information coordinator (CCIC) at their school. 
Coaches also were very satisfied or satisfied with the support they received from the ACES staff 
at MC. But several indicated that they were dissatisfied with the school resources (such as space 
or equipment). The majority of coaches also strongly agreed that they had someone in the school 
to provide support. Furthermore, one half of the coaches indicated they would like to see more of 
the following: cooperation between the program and the school, collaboration with the CCIC, 
flexibility in meeting with students, and just to be part of the school. 

All the staff survey respondents were either very satisfied or satisfied with the opportunities that 
the ACES program provided students and agreed or strongly agreed that overall, ACES met the 
needs of the students in the program. 
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To what extent did the ACES program increase student knowledge and assist in their 
participation of the college application and financial aid process? 

This section reports student and staff perceptions on the program’s assistance with the college 
application process. Data on college applications and acceptances, financial aid applications, and 
scholarship applications among all graduating ACES students will be in a subsequent ACES 
Year One outcome report. 

Student surveys. More than 80% of the students surveyed reported that they expect to 
receive a Bachelor’s degree or higher. Almost all student respondents agreed or strongly agreed 
that the ACES program assisted them with exploring colleges and universities, exploring majors 
and career interests, preparing them academically for college, and providing their parent or 
guardian with helpful information. Among the Grade 12 respondents, almost all agreed or 
strongly agreed that they were able to get the guidance they needed for the college application 
process and that the program assisted them in understanding the college application process and 
how to apply for financial aid and scholarships. Additionally, almost all senior respondents who 
were accepted to a 4-year college found the program very helpful to their gaining acceptance. 

ACES coach and resource counselor surveys. Most of the coaches strongly agreed, and 
most RCs strongly agreed or agreed, that ACES motivates students to go to college who might 
not have otherwise, helps students get accepted into a 4-year college, and helps students obtain 
financial aid. 

Recommendations 

During this first year of implementation, ACES staff, ACES coaches, and MCPS school staff had 
to establish procedures and roles for the program, create a comprehensive database for capturing 
student information, locate office space for the coach and meeting space for ACES activities, 
coordinate the location and schedule of student meetings, complete the recruitment of students, 
etc. Despite these first-year challenges, there were many successes. Surveyed students were very 
happy with the program and its guidance for college applications; staff also agreed that the 
program was successful. 

The following recommendations are to provide feedback for the program’s improvement and 
ongoing development and are based on findings from this part of the evaluation. 

• Continue to establish helpful and positive relationships between coaches and students. 

• Continue to build relationships between the ACES coach and school staff. 

o Areas of concentration, depending on specific school, include: procedures for sharing 
student information, coach’s use of equipment, attendance at counseling department 
meetings, clarification of roles for staff and for students, and regular communication 
between counseling staff and coaches. 

• Continue to provide clear and frequent communication with school staff members and 

ACES coaches about program updates and expectations. 
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• Explore providing more resources for coaches such as classroom and/or office space 
availability and equipment. 

• Explore scheduling of ACES activities to accommodate more students, 
o Consider duplicating important workshops and information sessions. 

o Consider altering coach’s schedules to more closely align to MCPS school schedules 
and increase coach’s availability (i.e., Fridays at school, spring breaks). 

• Explore possible misconceptions among some students and some counseling staff that 
students are required or encouraged only to attend MC and will not receive help applying 
to a 4-year college. 

• Consider adding more college field trips, especially early in the school year, and 
information regarding scholarship opportunities. 

• Schedule time for ACES school-level teams from the 10 schools to come together to 
clarify roles and responsibilities and share best practices with each other. 
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Formative Evaluation of ACES Program: Findings from Surveys and 
Interviews - Year One, Grades 11 and 12 

Natalie Wolanin and Shaphar Modarresi, Ph.D. 


The Office of Shared Accountability (OSA) in Montgomery County Public Schools (MCPS) is 
conducting a multiyear evaluation of the Achieving Collegiate Excellence and Success (ACES) 
program. The year one study uses a multimethod design to conduct formative evaluations as well 
as immediate outcome or output evaluations. This is the first of a series of reports provided for 
the first year of the ACES program. This report focuses on a portion of the formative evaluation, 
the goal of which is to document how well the ACES program components are being 
implemented and to provide feedback to relevant stakeholders for the program’s improvement as 
well as its ongoing development. ACES student and staff perceptions on their experiences and 
satisfaction with the first year of the program were collected through surveys and interviews and 
are described here. 


Background 

MCPS collaborated with Montgomery College (MC) and the Universities at Shady Grove (USG) 
to create the ACES program. The ACES program seeks to create a seamless pathway from high 
school to college completion. It focuses on identifying and supporting both students who are 
underrepresented in higher education and those who would be the first in their family to attend 
college. 

The first year evaluation of the ACES program focused on students in Grades 11 and 12 in 
MCPS schools with the ACES program who met one or more of the following risk factors: low 
income or single parent household; member of an underrepresented race/ethnicity group in 
higher education (such as Black or African American or Hispanic/Latino students); first 
generation college student; students with disabilities or in special education; immigrant or child 
of immigrant parents; homeless students or those living in unstable conditions. For the 2013- 
2014 school year, students had to apply to the ACES program and were considered eligible for 
acceptance if they met at least one of the criteria listed above. 

The ACES program provides the following services to students in Grades 11 and 12 (see 
Appendix B for full flyer): 

• Academic coaching 

• Career exploration and how to choose a college major 

• College tours 

• Preparation for college placement tests 

• Individual assistance with college applications and financial aid 

• Information sessions for parents 

• Summer programs at MC and the USG 

• Transition services to college 
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A central element to the ACES program is the presence of coaches who mentor, advocate, and 
advise ACES students. They work one-on-one with students on how to be successful in high 
school, completing college admission applications, negotiating the scholarship and financial aid 
process, and transitioning from high school to college, as well as providing weekly activities, 
group meetings, and college trips. Parents also were invited to attend select workshops such as 
completing financial aid paper work and applying for scholarships. An ACES coach, employed 
by MC, works at each of the 10 participating high schools four days a week, where they meet 
with ACES students and coordinate college readiness activities. Coaches work with the 
counseling department in their school to manage the ACES program and share student 
information. These program activities are in addition to, and align with, the college and career 
supports provided by MCPS staff members. Students are expected to commit to their own 
academic success and participate in planned activities, as well as to meet with their ACES coach 
on a regular basis. 

The next phase of the ACES program occurs after graduation from high school. If students 
choose to attend MC and the USG, they will receive continued support from an ACES academic 
coach at these colleges. However, the goal of the ACES program is for students to complete a 
4-year degree. Therefore, students do not need to plan on attending MC and the USG to 
participate in the high school ACES program. 

Eight high schools were initially identified to implement the ACES program, and schools began 
recruiting students in the spring of 2013. At the beginning of the 2013-2014 school year, two 
additional high schools were added to implement the ACES program, for a total of ten schools. 
All the ACES coaches were hired during the summer and began working at the schools in the fall 
of the 2013-2014 school year. 

The 10 MCPS schools participating in the ACES program are: Montgomery Blair, Clarksburg, 
Albert Einstein, Gaithersburg, John F. Kennedy, Northwood, Rockville, Seneca Valley, 
Wheaton, and Watkins Mill high schools. 

Evaluation Questions 

The following are the formative questions for the Year One study that were developed in 
collaboration with ACES program administrators. 

1. Who were the students enrolled in year one of the ACES program? 

2. To what extent are the program activities and/or features as proposed in the program 
design being implemented at schools, and how consistent is the level of implementation 
across schools? 

3. To what extent do students participate in program activities? 

4. How is the ACES program perceived by students and program staff? 

5. To what extent did the ACES program increase student knowledge and assist in 
their participation of the college application and financial aid process? 

This report addresses the formative evaluation questions 4 and 5. The remaining formative 
evaluation questions, as well as outcome questions, will be addressed in subsequent reports. 
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Literature Review 

The Advisory Committee on Student Financial Assistance (2006) stated that because attainment 
of a bachelor’s degree is essential to America’s economic future, narrowing income-related gaps 
is of critical importance. Doing so requires that college-qualified high school graduates from 
low- and moderate-income families have the financial resources to enroll and persist through 
degree completion. However, our nation is nowhere near achieving this goal, and significant gaps 
in the attainment rate of a bachelor’s degree by family income continue to exist. 

A study done by the National Center for Education Statistics (NCES) in 2012 on higher 
education and access gaps reported: 

“Many studies have documented the associations between student background 
and educational outcomes (e.g., high school graduation rates and postsecondary 
enrollment, persistence, and attainment rates). Demographic factors known to be 
linked to these outcomes include socioeconomic status indicators (e.g., poverty, 
family income, and parents’ education), parental involvement, student disabilities, 
and native language.” (p. 7) 

The study explains that, “poverty poses a serious challenge to a child’s ability to 
succeed in school and its prevalence is markedly higher among certain 
racial/ethnic groups than in others. Research has suggested that living in poverty 
in early childhood is associated with lower than average academic performance 
that extends through elementary and high school and can lead to lower than 
average rates of [high] school completion.” (p. 7) 

The study also cites research by Nunez and Cuccaro-Alamin (1998) who report that, 

“Among beginning postsecondary students in 1989-90, first-generation college 
students (college students whose parents had never enrolled in postsecondary 
education) persisted in postsecondary education and attained credentials at lower 
rates than their non-first-generation counterparts. This finding held for students at 
4-year institutions as well as public 2-year institutions. Even when controlling for 
many of the characteristics that distinguish first-generation college students from 
their peers, such as socioeconomic status, institution type, and attendance status, 
first-generation student status still had a negative effect on persistence and 
attainment.” (NCES, 2012, p. 7) 

MCPS, using StudentTracker Data from the National Student Clearinghouse, examined 
graduated students from 2001 to 2010 and found that in total, 70% of MCPS graduates went to 
college in the fall immediately after high school: 47% went to 4-year institutions, and 23% went 
to 2-year institutions (Zhao, 2012). The University of Maryland at College Park received the 
most MCPS graduates among 4-year institutions, and Montgomery College at Rockville enrolled 
the most MCPS graduates among 2-year institutions. Among the 2001 to 2010 high school 
graduates who received Free and Reduced-price Meals System (FARMS) services in Grade 12, 
51% went to college in the fall; 43% of graduates who received special education services in 
Grade 12 went to college in the fall; and 34% of graduates who received English for Speakers of 
Other Languages (ESOL) in Grade 12 went to college in the fall. Fifty-eight percent of African 
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American graduates and 50% of Hispanic graduates went to college in the fall; this compared to 
78% among Asian graduates and 79% among White graduates. 

The ACES Task Force studied best practices of similar programs including the Pathway to the 
Baccalaureate program (MCPS, MC, and USG, 2012). This program is a consortium of 
10 educational institutions in Northern Virginia consisting of K-12 public school systems, 
Northern Virginia Community College (NOVA), and George Mason University. Its goal is to 
support students as they make the transition from high school through NOVA to George Mason 
University or another public university in Virginia. 

Fairfax County Public Schools (FCPS) has the highest participation in the Pathway to the 
Baccalaureate program (FCPS, 2013a). Program services are provided onsite at participating 
high schools during regular school hours and include a one-stop case management model for 
student services (e.g., placement testing, academic advising, and early course registration). The 
program also includes financial aid/literacy training and the availability of program specific 
grants and scholarships; a learning community/cohort-building model beginning in high school; 
and an early, continuous investment in student success by all participating institutions. Reports 
show that 86% of students in the program’s first six cohorts have transitioned from high school 
directly into postsecondary education and 90% of students in the program at NOVA are retained 
from the first to the second semester (FCPS, 2013b). 

Methodology 


To answer the two evaluation questions, this study utilized a multimethod data collection 
strategy, including staff surveys (resource counselors [RCs] and ACES coaches), student 
surveys, and midyear staff interviews (RCs and ACES coaches). The evaluators, in collaboration 
with ACES administrators, developed instruments for these activities. Many of the survey 
questions were developed from information gathered through the midyear interviews, interviews 
with ACES administrators, and relevant literature. 

Data was collected from ACES coaches, RCs, and ACES students within the 10 participating 
schools. 

Data Collection 
Student Surveys 

The purpose of the student survey was to collect information pertaining to the experiences of 
students who were in the ACES program. The surveys also were designed to capture ACES 
implementation challenges, benefits to students, and areas for improvement. 

An anonymous online student survey was offered at the end of the 2013-2014 school year. 
ACES coaches were sent the link to the survey in May and were asked to direct all participating 
students to complete the survey before the end of the school year. Several reminders were sent to 
coaches to increase the response rate. There were a total of 968 students in the ACES program 
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the first year, and 474 students responded to the survey (a 49% response rate). A breakdown of 
the response rates by grade and school may be seen in Table 1. For example, there were 406 
students in Grade 1 1 in the ACES program and 204 responded, for a 50% response rate among 
Grade 11 ACES students. The schools’ response rates ranged from 19% to more than 100%. One 
school had more than a 100% response rate because five more surveys than students enrolled in 
the program were submitted at that school. It is speculated that several students submitted more 
than one survey; however, due to the anonymity of the survey, the duplicate surveys could not be 
identified and all data was included. 


Table 1 


Student Survey: Number and Percentage of Respondents and 
Response Rates by Grade Level and School 



Response Rates per Grade 
Level and School 
(N = 968) 

N n % 

Total 3 

968 474 49.0 

i Grade 1 1 

Grade level _ , 

Grade 12 

406 204 50.2 

562 270 48.0 

Blair 

90 56 62.2 

Clarksburg 

85 90 105. 8 b 

Einstein 

92 17 18.5 

Gaithersburg 

112 29 25.9 

Kennedy 

School XT , , 

North wood 

69 34 49.3 

122 87 71.3 

Rockville 

104 53 51.0 

Seneca Valley 

116 31 26.7 

Wheaton 

59 19 32.2 

Watkins Mill 

119 55 46.2 


Note. Three student respondents did not indicate their school. 

“Some students who are in Grade 10. based on completion of specific courses, are 
included in the Grade 11 count here. 

b More students completed the survey than enrolled; see discussion in text. 


Staff Surveys 

The purpose of the staff surveys was to collect information pertaining to the experiences of 
school staff who were implementing the program at the school level. The surveys also were 
designed to capture ACES implementation challenges, benefits to students, and areas for 
improvement. 


An anonymous online survey was offered at the end of the 2013-2014 school year to all 
10 ACES coaches and all 10 RCs in the participating schools. All 20 staff members completed 
the survey for a 100% response rate. 


Staff Interviews 


The purpose of the staff interviews was to gain a comprehensive perspective about coaches’ and 
RCs’ experiences and opinions pertaining to the implementation of ACES during the first several 
months of the program. 


Program Evaluation 


CONFIDENTIAL 5 


DRAFT 


Evaluation of ACES Year One 



Montgomery County Public Schools 


Office of Shared Accountability 


All 10 ACES coaches and all 10 RCs in the participating schools were interviewed by evaluators 
in January 2014, and interviews were performed face-to-face, employing a structured 
questionnaire. Interview findings were presented in an internal MCPS memorandum dated 
April 2014 and provided evaluators with guidance for creating the end-of-year surveys. Many of 
the interview findings are mentioned in this report where applicable. 

Analytical Procedures 
Qualitative Data 

Information gathered through interviews and open-ended survey questions was analyzed through 
content analysis. Researchers looked for themes in the data, and where appropriate, category 
frequencies were reported. In addition, responses to open-ended interview and survey questions 
provided examples and context for the quantitative information presented in the findings. 

Quantitative Data 

Quantitative information was compiled from closed-ended survey responses; analyses were 
conducted using descriptive statistics. 

Strengths and Limitations 

In drawing conclusions from the current study, several strengths and caveats must be noted. 


Strengths 

First, the existing program documents, as well as initial program staff interviews were used to 
guide development of survey items for coaches, RCs, and students. This strategy improved the 
construct validity of measures by developing a set of survey items (or questions) that were both 
relevant and valid. 

Second, before administering the student surveys, the questionnaire was examined by several 
ACES staff members to ensure that the questions were appropriate and clear. Furthermore, the 
survey instrument was piloted by a few ACES students to reduce measurement errors using the 
following types of questions as recommended by Isaac & Michael (1995): Are the 
directions/instructions for the survey clear? Is the language of the questions appropriate for high 
school students? Are the questions easy to understand? Are the response choices to the questions 
appropriate? Are the questions leading in any way — i.e., do they suggest a particular way to 
answer? Do any of the questions ask sensitive information that might make a student 
uncomfortable? Do any of the questions seem unimportant to a study of the ACES program? Are 
there questions about the ACES program that should be added? Does the survey seem too long or 
too short? 


Program Evaluation 


CONFIDENTIAL 6 


DRAFT 


Evaluation of ACES Year One 



Montgomery County Public Schools 


Office of Shared Accountability 


Third, all 10 ACES coaches and all 10 RCs were interviewed in January, and all completed an 
end-of-year survey (100% response rate) ensuring the external validity of findings from coaches 
and RCs. 

Finally, this study addressed the evaluation questions by means of cross-method comparisons. 
Although the interview and survey data were collected independently, they still focused on the 
primary objectives of the evaluation. Therefore, convergent findings between the interviews and 
surveys in some instances increase the validity of findings. 

Limitations 

First, at the system level, about half of the ACES students did not respond to the survey 
questions (response rate = 49%; n = 474), therefore, the generalization of the student survey 
results (external validity) is limited only to those students who responded. Second, some schools 
had a much lower representation of students in the student survey data than others. Due to 
variation in the school-level response rate, caution should be exercised when viewing these 
findings in the context of an individual school. Third, 18 of the 474 student surveys (from a 
variety of schools) were partially completed. The survey data of those students were still 
included in the analyses because many of the survey questions were answered by students. In 
addition, one school had five more surveys (including three partially completed) than the total 
number of students enrolled in the program, yielding more than a 100% response rate. It is 
speculated by the evaluators, but unknown, that several students submitted more than one 
survey. Due to the anonymity of the surveys, the duplicates could not be identified and all data 
were included in the analyses. Finally, causality cannot be inferred from the findings because 
the data are from an uncontrolled study. In contrast to controlled experiments, studies that are 
based on survey designs are inherently uncontrolled and lack strong internal validity. 


Findings 

Student Enrollment in ACES 
First Learned About ACES Program 

Students were able to select more than one response to the question addressing ways that the 
student first learned about the program. As shown in Table 2, more than one third (35%) of 
responding students reported they were contacted by a school counselor about the ACES 
program and more than one fourth (27%) said they learned about ACES during class or were 
contacted by the ACES coach (26%). Additionally, one fifth (20%) reported they learned about 
the program during an assembly. 
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Table 2 

Student Survey: Ways that Students First Learned about the ACES Program 


A =474 



n 

% 

Contacted by a school counselor 

165 

34.8 

During a class 

129 

27.2 

Contacted by the ACES coach 

122 

25.7 

During an assembly 

95 

20.0 

Contacted by a teacher 

60 

12.7 

From another student 

57 

12.0 

A flier or bulletin 

49 

10.3 

Contacted by the college and career information coordinator 

33 

7.0 

Contacted by an administrator 

15 

3.2 

Other: mother/parent 

5 

* 

Other (e.g., AVID program, mail, lunch. Connect Ed) 

16 

3.4 


Note. More than one response could be chosen. 
*Less than 1%. 


Recruiting ACES Students 

In interviews, ACES coaches and RCs indicated that the RC, many in partnership with the other 
school counselors, provided the initial list of participating ACES students to the ACES coach 
upon their arrival in the fall. For most of the schools, the ACES coach did additional recruiting in 
the fall to increase the number of students enrolled. In the case of the two schools added to the 
program in the fall, the ACES coach did the bulk of the recruiting. 

Both interviews and online surveys of the same staff members found that various methods for 
recruiting students were used across the schools. Examples of recruiting methods used were: 
meeting with potential students, referrals from counselors to coaches, presenting the ACES 
program to students in English class, holding an ACES assembly, informally approaching 
students, advertising at the school or using the Connect-ED all-call program, mailing 
applications home, utilizing the support of the Parent Teacher Student Association, and obtaining 
additional student referrals from teachers and counselors. 

According to interviews with coaches and RCs, all the students who were in the program were 
accepted because they met at least one of the ACES criteria of being underrepresented. The 
acceptance criteria included but were not limited to underrepresented students in college 
including: those receiving FARMS services, Black or African American or Hispanic/Latino, first 
generation college student, or immigrant. Interviews revealed that most students who applied 
were accepted because they were recruited, and the few that were not accepted did not meet any 
of the qualifying criteria. Several interviewees explained their specific strategies for enrolling 
students, such as including everyone in the Advancement Via Individual Determination (AVID) 
program, recruiting students in ESOL classes, and recruiting students receiving FARMS services 
while also eliminating students with attendance issues or those not on track to graduate. A 
couple of interviewees also indicated that a grade point average (GPA) minimum was not 
required. 
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Evaluation Question: How is the ACES program perceived by students and program staff? 
Student Responses 

Reasons for applying to the ACES program. As shown in Table 3, more than two thirds 
of student respondents selected answers related to preparing for college as reasons for applying 
to the program including: support preparing for college, such as standardized testing, study skills, 
etc. (69%); assistance with applying to college (68%); assistance with applying for scholarships 
(67%); assistance with applying for financial aid (67%); and to explore options for attending 
college (67%). More than one half (56%) of the students responding reported that they liked the 
opportunities that ACES offers. 


Table 3 


Student Survey: Reasons for Applying to the ACES Program 



N = 

474 


n 

% 

I wanted support with preparing for college (such as: SAT/ACT, 
ACCUPLACER, study skills, course selection, etc.). 

328 

69.2 

I wanted assistance with applying to college. 

324 

68.4 

I wanted assistance with applying for scholarships. 

319 

67.3 

I wanted assistance with applying for financial aid. 

318 

67.1 

I wanted to explore options for attending college. 

315 

66.5 

I liked the opportunities that the ACES program offers to students. 

263 

55.5 

I wanted one-on-one/individual guidance. 

203 

42.8 

I wanted to continue to have a coach guide me after high school. 

129 

27.2 

My counselor/parent told me to apply. 

94 

19.8 

Other (AVID required, heard about coach, unspecified). 

4 

* 


Note. More than one response could be chosen. 


Experience with ACES. Students were asked to rate a series of statements to address their 
experiences with the ACES program. As shown in Table 4, almost all ACES student respondents 
agreed or strongly agreed that they would recommend the program to other students (99%), that 
the coach provided them with helpful information and guidance (99%), that the coach was 
available to meet if needed (98%), that the application process was easy (97%), that the 
workshop topics were useful (97%), that they looked forward to ACES activities (96%), that 
participation was important to achieve goals (96%), and that ACES has motivated them to go to 
college (93%). A large majority (86%) agreed or strongly agreed that the workshop times 
available were convenient; however, less than one third (32%) strongly agreed with this 
statement. 
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Table 4 

Student Survey Respondents’ Experience with ACES 



Strongly 





Strongly 


Agree 

Agree 

Disagree 

Disagree 

Survey Items 

n 

% 

n 

% 

n 

% 

n 

% 

I would recommend the ACES program to other 
students. ( N = 465) 

319 

68.6 

141 

30.3 

3 

0.6 

2 

0.4 

The ACES coach provided me with helpful information 
and guidance. ( N = 463) 

296 

63.9 

161 

34.8 

4 

0.9 

2 

0.4 

The ACES coach was available to meet if I needed any 
assistance or information. (N = 461) 

289 

62.7 

164 

35.6 

5 

1.1 

3 

0.7 

The ACES application process was easy for me. 
(At =464) 

241 

51.9 

210 

45.3 

9 

1.9 

4 

0.9 

Overall, the ACES workshop topics were useful to me. 
(N = 449) 

187 

41.6 

248 

55.2 

10 

2.2 

4 

0.9 

Overall, I look forward to ACES activities, such as the 

211 

47.0 

219 

48.8 

15 

3.3 

A 

0.9 

workshops, meeting with coach, etc. ( N = 449) 


Participation in the ACES program is important for me 
to achieve my future goals. ( N = 464) 

227 

48.9 

216 

46.6 

19 

4.1 

2 

0.4 

The ACES program has motivated me to go to college. 
(At =444) 

233 

52.5 

180 

40.5 

25 

5.6 

6 

1.4 

Overall, the times available for the ACES workshops 
were convenient for me. (N = 448) 

145 

32.4 

241 

53.8 

53 

11.8 

9 

2.0 


Using a 5 -point scale, more than one half of the students surveyed rated their overall experience 
as excellent (58%) and the overall quality of the program as excellent (60%), as shown in Table 
5. Approximately one third gave a rating of good for overall experience (35%) and overall 
quality of the program (34%). 


Table 5 

Student Survey Respondents’ Overall Experience with ACES 


Excellent 

Good 

Average 

Poor 

Very Poor 

n % 

n % 

n % 

n % 

n % 

Overall experience (N = 452) 260 57.5 

156 34.5 

32 7.1 

1 0.2 

3 0.7 

Overall quality of the program 

(At = 445) 267 60.0 

151 33.9 

25 5.6 

0 0.0 

2 0.4 


Positive aspects of the ACES program. A vast majority (84%) of students reported that 
the ACES coach is what they liked about the program (Table 6). Almost two thirds or more also 
reported that they liked receiving information on scholarships and financial aid (69%), 
information on colleges and universities (68%), and meeting individually with their coach (65%). 
More than one half (57%) also reported that they like receiving information on careers and 
majors. Less than half identified workshops, field trips, or connecting with other students as what 
they liked. 


Program Evaluation 


CONFIDENTIAL 


10 


DRAFT 


Evaluation of ACES Year One 



Montgomery County Public Schools 


Office of Shared Accountability 


Table 6 


Student Survey: Aspects of the ACES Program which Students Liked 



N = 
n 

474 

% 

ACES coach 

399 

84.2 

Information Scholarships/Financial Aid 

326 

68.8 

Information on Colleges/Universities 

322 

67.9 

Meeting individually with Coach 

309 

65.2 

Information on careers/majors 

271 

57.2 

ACES workshops 

225 

47.5 

Field Trips 

184 

38.8 

Connecting with other ACES students 

166 

35.0 

Other 

3 

* 


Note. More than one response could be chosen. 
*Less than 1%. 


Challenges with the ACES program. One half of the student respondents reported that 
the workshops were at times they could not attend (50%), as shown in Table 7. One fourth 
reported that they found it hard to find time to meet with the coach (25%) and to keep motivated 
to participate in the program (25%). Finally, one fifth (20%) reported that they were not sure 
whether to go to a counselor, ACES coach, or College and Career Information Coordinator 
(CCIC) for assistance. 


Table 7 

Student Survey: Challenges with ACES 



N = 

474 


n 

% 

Workshops at times unable to attend 

239 

50.4 

Hard to find time to meet with ACES coach 

120 

25.3 

Keeping motivated to participate 

116 

24.5 

Not sure whether to go to ACES coach, counselor, or CCIC for 
assistance 

93 

19.6 

Other (e.g., overwhelmed, participation, coach’s demeanor, not 
enough help, etc.) 

9 

1.9 


Note. More than one response could be chosen. 


Students ’ suggestions for the ACES program. In an open-ended question where students 
could offer suggestions to improve the ACES program, 17% (n = 26) of the 152 students who 
offered suggestions stated that they would like to see more trips to colleges and field trips in 
general (Table A1 in Appendix A). Approximately 15% (n = 22) made a suggestion related to 
accommodating their schedule, such as offer more convenient times, more times available, more 
flexibility, more times during the day, etc. Just under 10% gave the following suggestions: offer 
more individualized and one-on-one assistance (n = 15, 10%); have a more motivated and 
involved coach (n = 13, 9%); offer more scholarships opportunities and information (n = 13, 
9%); offer other topics such as tips on college and tips on personal finance and careers (n = 11, 
7%). One student suggested to “make it clear you won’t send people straight to Montgomery 
College. A lot of junior guys that need assistance didn’t join as they were under the impression 
you had to go to MC if you joined.” This is a misconception worth exploring further and 
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coincides with a survey finding among a few counseling staff who disagreed that the ACES 
program helps students get accepted to a 4-year college. 

Gratitude for the ACES program. Students had the opportunity to leave a comment at the 
end of the survey. Of the 149 students who left comments, 95% were either positive towards the 
ACES program, sent a thank you to the coach or ACES program, or both. Table 8 shows some 
examples. 


Table 8 

Examples of Student Survey Comments (Open-ended) 

“ACES pushed me to be a better student and strive to meet goals.” 

“Since I only live with my mom, and she only attended a few weeks of community 
college, ACES has been extremely helpful in guiding me in what 1 should be 
doing/looking for in the college process. Since my mother doesn’t know how to help 
me, my ACES coach has been so great to have! 1 am looking forward to it next year! 
And since 1 will be a senior next year, my coach will be so much more of a help!” 

“The ACES program pushed me to want to get more involved with my future.” 

“1 am so glad and very relieved that 1 am in the program. 1 feel it has helped me so 
much since the first day. 1 don't think 1 would have been able to do everything by 
myself if I wasn’t in this program. I do want to thank the coach for helping me with 
everything.” 

“The ACES coach is very helpful and respectful. The coach takes the time to call you 
individually, to get to know you better and to understand your situation better. The 
coach does everything that they can to keep you motivated and see you succeed in 
school. 1 think that the ACES program is very helpful based on my experience.” 

“It made me realize that there are so many teens like me who really need this 
program. All the questions we did not get a chance to ask while we were in class, we 
were able to ask those questions there and not feel out of place.” 


Staff Responses 

Although the staff surveys yielded a 100% response rate, there are only 10 ACES coaches and 
10 RCs; therefore, only numbers, not percentages, are shown in the survey findings. 
Additionally, midyear interviews were conducted with these coaches and RCs. Relevant 
findings from the interviews are reported throughout. 

Ways in which coaches and counseling staff work with each other. In the online survey 
given at the end of the school year, ACES coaches were asked to self-report the ways in which 
they worked with the RC, the other counselors, and the CCIC at the school. The RCs also were 
asked their perspective of how the ACES coaches worked with the same school staff. As shown 
in Table 9, the majority of coaches and RCs reported that the coaches provided updates on ACES 
activities to the RC, counselors, and CCIC ( n = 16 or 17); the coaches provided updates about 
ACES students to the RC, counselors, and CCIC ( n = 13 or 16); and the coaches exchanged 
information about students with the RC, counselors, and CCIC ( n = 14 or 16). 

Although 15 of the 20 staff members reported that coaches worked with the CCIC on college 
plans, more RCs reported this than coaches (n = 9 and n = 6, respectively.) Similarly, of the 
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13 staff members who reported that the coach collaborates on students’ college plans with the 
RC, more RCs reported this than coaches (n = 8 and n = 5, respectively). 

One half, to just under one half, of the staff members surveyed reported that the coach 
coordinates the planning of ACES workshops, activities, and assemblies with the CCIC ( n = 10) 
and RC (n = 8). One fourth reported this coordination with other counselors, and four of the five 
who reported this were coaches. Finally just under one half reported that the coach collaborated 
with the RC and/or the CCIC on parent communications ( n = 9). 

In interviews conducted in January, all of the RCs stated that they (or the other counselors) 
worked with the coach on a regular basis. A variety of ways in which they met were reported: 
informally, as needed, 2-3 times a week, and at a regular weekly meeting. Working together on 
caseloads, assisting the coach with getting in touch with students and resources, and coordinating 
activities also were examples that were reported by RCs. 


Table 9 

ACES Coach and RC Survey: How ACES Coach works with School Staff 



How Coach works with. . 




Other 



RC 

Counselors 

CCIC 


(N = 20) 

(N = 20) 

(N = 20) 

Provide updates on activities offered through 
ACES 

17 

16 

16 

Provide updates about students in the ACES 
program 

16 

16 

13 

Exchange information about ACES students 

16 

14 

16 

Collaborate on students’ college plans 

13 a 

10 

15 c 

Coordinate planning of ACES 
workshops/activities/assemblies 

8 

5 b 

10 

Collaborate on parent communications 

9 

6 

9 

Not applicable, does not work with RC/other 
counselors/CCIC 

1 

0 

1 

Other 

3 

3 

3 


a Five coaches and eight RCs 
“Four coaches and one RC 
“Six coaches and nine RCs 


Attendance at counseling meetings. Almost all coach survey respondents reported that 
they attended counseling department meetings either regularly ( n = 5) or sometimes in = 4). The 
reasons given for not attending meetings were: they have to be at a MC meeting; the information 
at the counseling meeting is not related to ACES; or they are not included in the meetings. 

Collaboration between staff. Almost three fourths (n = 14) of RCs and coaches reported 
that collaboration between the ACES coach and RC was successful with no challenges; four 
reported some challenges; and two reported it was difficult (Table 10). Most respondents were 
split about whether the processes for coaches and school staff members to work on students’ 
college plans and for the coach to provide regular feedback about students and ACES services 
and progress were successful with no challenges or some challenges. Two respondents reported 
that each of these processes was difficult. The coaches’ reports on collaboration between the 
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CCIC and other school counselors varied, with four reporting collaboration with the CCIC was 
difficult. 


Table 10 


ACES Coach and RC Survey: Experience with Other Staff in 

ACES Program 



Successful 
with no 
challenges 

Successful 
with some 
challenges 

Difficult 
implementation 
. .need to address 
challenges 

Resource Counselors and Coaches ( N = 20) 

n 

n 

n 

Collaboration between the coach and RC 

14 

4 

2 

Process for coaches to provide regular feedback about the services 
students are receiving/progress students are making 

10 

8 

2 

Process for ACES coach and school staff to work on students’ 
college plans 

8 

10 

2 

Coaches (N = 10) 

Collaboration between the coach and CCIC 

5 

1 

4 

Collaboration between the coach and other school counselors 

3 

6 

1 


Support for ACES Coaches 

Access to student data. In interviews with coaches and program staff (i.e., ACES 
co-chairs and director) earlier in the year, it was mentioned that some coaches did not have easy 
access to student data. However, when asked in the survey about access to student data, almost 
all coaches ( n = 9) reported that they received student data through their access to the Online 
Administrative Student Information System (OASIS), and 7 of 10 also reported that they ask the 
counselor for student information or ask the student directly (Table 11). 


Table 1 1 

ACES Coach Survey: Access to Student Data ( N = 10) 


Access to Student Data 

n 

Through my access to OASIS 

9 

Ask counselor for the student information 

7 

Ask student directly 

7 

Through my access to Naviance 

6 

Through my access to FileMaker 

5 

Not applicable, do not receive data on students 

0 

Other: College Track, OASIS through counselor’s help 

2 


Note. More than one response could be chosen. 


When both RCs and coaches were asked a different survey question about the coach’s access to 
student data, the responses varied between “successful with no challenges,” “successful with 
some challenges,” and “difficult implementation. . .need to address challenges” (Table 12). 
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Table 12 


ACES Coach and RC Survey: Respondents’ Experience with Access to Data 



Resource Counselors (N = 10) 

ACES Coaches ( N 

= 10) 


Successful 
with no 
challenges 

Successful 
with some 
challenges 

Difficult 
implement- 
tation. . .need to 
address 
challenges 

Successful 
with no 
challenges 

Successful 
with some 
challenges 

Difficult 
implemen- 
tation. . .need 
to address 
challenges 


n 

n 

n 

n 

n 

n 

Coach’s access to student 
data as needed (i.e., GPA, 
grades, test scores, 
demographics) 

3 

6 

i 

5 

2 

3 


Coach satisfaction with supports. All or almost all of ACES coaches were very satisfied 
or satisfied with the professional development they received for their position, support from the 
staff at MC, and support from their assigned school (Table 13), with eight reporting they were 
very satisfied with the support from the staff at MC. Four coaches reported that they were 
dissatisfied or very dissatisfied with the school resources available such as meeting space, office 
space, and equipment or use of equipment. 


Table 13 


ACES Coach Survey: Satisfaction with ACES Program 



ACES Coaches (N = 10) 



Very 

Satisfied 

Satisfied 

Dissatisfied 

Very 

Dissatisfied 


n 

n 

n 

n 

The professional development I have received 
for my position as ACES coach 

4 

6 

0 

0 

School resources available for ACES (space for 

9 

A 


1 

group meetings, office space, equipment) 

z 

‘-r 

D 

Support I’ve received from the ACES staff at 

Q 

9 

0 

0 

Montgomery College 

O 

Z 

Support I’ve received from my assigned school 

2 

7 

1 

0 


All or almost all ACES coaches agreed or strongly agreed that they have someone at the school 
they can go to with questions ( n = 10), and they have the support they need to run the program at 
their school ( n = 9). 

Coaches ’ caseload. When asked, most coaches reported that their caseload included too 
many students in = 7) or just the right amount of students (n = 3). 

Support needed for coaches. ACES coaches were asked, in an open-ended question, what 
supports they still needed (responses are shown in Table A2, Appendix A). Five indicated more 
cooperation between the program and the school such as schoolwide awareness and recmitment, 
collaboration with the CCIC, flexibility in meeting with students, and just to be part of the 
school. Five also indicated a need for resources such as a space with several computers to hold 
meetings with multiple students and more available school resources. Finally, a few coaches 
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stated a need for access to data, referring to either MCPS student data or a full functioning ACES 
database to enter comprehensive student information. 

Implementation and Success of Program 

Success of program aspects. In the staff surveys, both ACES coaches and RCs were 
asked about the level of success implementing various aspects of the program this first year 
(some of these findings were reported above). Most RCs in = 7) reported that the process for 
recruiting students and the enrollment process were successful with some challenges, whereas 
the coaches were divided on these areas, with three reporting that the process for recruiting 
students was difficult (Table 14). Five of the RCs reported that compiling and managing records 
was successful with no challenges; two reported it was not implemented. Six coaches reported 
compiling and managing records was successful with some challenges, and three reported it was 
difficult. Finally, almost all coaches ( n = 9) reported that the process for offering workshops to 
ACES students or for meeting with students individually was successful with either no 
challenges or some challenges. 


Table 14 

ACES Coach and RC Survey: Respondents’ Experience with Implementation of 
ACES Program 



Resource Counselors (N = 10) 

ACES Coaches (N = 10) 


Successful 
with no 
challenges 

Successful 
with some 
challenges 

Difficult 
implementation 
. . .need to address 
challenges 

Successful 
with no 
challenges 

Successful 
with some 
challenges 

Difficult 

implementation. . . 
need to address 
challenges 


n 

n 

n 

n 

n 

n 

Process for recruiting 
students into the ACES 
program 

2 

7 

i 

3 

4 

3 

The enrollment process for 
the ACES program 

2 

7 

i 

5 

4 

1 

Compiling and managing 
records of all ACES student 
activities and participation 

5 

3 

o a 

1 

6 

3 

Process for offering 
workshops to ACES 
students 

n/a 

n/a 

n/a 

3 

6 

1 

Process for meeting with 
students individually 

n/a 

n/a 

n/a 

4 

5 

1 


a Two selected “not implemented.” 


Clarity of program aspects. From the end-of-year surveys, almost all RCs and coaches 
agreed or strongly agreed that the goals of the program were clear, their role with the ACES 
program was clear, and the responsibilities of the students were clear (Table 15). Four RCs 
disagreed that they’ve been kept informed about ACES program decisions and changes to the 
program, and two disagreed that their role with the program is clear. Midyear interviews with 
coaches and RCs revealed that many felt that more clarity of roles between coaches and school 
counseling staff was needed. Additionally, many felt that communicating to the RCs and school 
staff members about program goals and program updates was needed. 
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Table 15 

ACES Coach and RC Survey: Respondents’ Experience with ACES Program 



Resource Counselors (N = 10) 

ACES Coaches (N 

= 10) 


Strongly 

Agree 

Agree 

Disagree 

Strongly 

Agree 

Agree 

Disagree 


n 

n 

n 

n 

n 

n 

The goals of the ACES program are clear. 

4 

5 

i 

4 

6 

0 

My role with the ACES program is clear. 

3 

5 

2 

5 

5 

0 

Responsibilities of the students in the ACES 
program are clear. 

3 

6 

1 

3 

7 

0 

1 have been kept informed about decisions and 
changes to the implementation of the ACES 
program. 

4 

2 

4 

5 

5 

0 

The ACES coach is able to provide additional 
support to students in the program. 

7 

3 

0 

n/a 

n/a 

n/a 

1 have been able to provide adequate one-on- 
one time with students who need it. 

n/a 

n/a 

n/a 

5 

4 

1 

I am able to address the needs of the students in 

n/a 

n/a 

n/a 

A 

f. 

0 

the ACES program. 

T- 

u 

Overall, ACES has met the needs of the 
students in the program during school year 
2013-2014. 

5 

5 

0 

7 

3 

0 


Meeting program goals. All RCs agreed (n = 3) or strongly agreed (n = 7) that the 
coaches were able to provide additional support to students in the program (Table 15). Most all 
or all of the coaches agreed or strongly agreed that they have been able to provide adequate one- 
on-one time to students who need it and have been able to address the needs of students in the 
program. All RCs and coaches agreed or strongly agreed that overall, ACES met the needs of 
the students in the program. 

When ACES coaches and RCs were asked how satisfied they were with the opportunities the 
program provides students, 9 reported very satisfied, and 1 1 reported satisfied. 

Positive aspects of the ACES program. Coaches were asked in an open-ended question, 
“What is going well?” Categories and examples of responses are shown in Table A3 of 
Appendix A. One half or more commented that school and staff support plus communication 
were going well ( n = 6), and individual meetings with students and/or parents (n = 5) were going 
well. Four RCs reported that the collaboration and communication from the ACES coach and 
the support and interaction that the coach provided to students was going well. 

Resource counselors were asked the same open-ended question (Appendix A, Table A4). Only 7 
of the 10 RCs left a comment. Four reported collaboration and communication from the ACES 
coach was going well and that student support and interaction by the coach was going well. 

In interviews conducted in January, there was a wide variety of comments about what was going 
well. The top themes that emerged were: relationships among coaches, counselors, and school 
staff (14 of 20 interviewees); work being done with students (12 of 20 interviewees); 
relationships among coaches and students (7 of 20 interviewees); and workshops (7 of 20 
interviewees). 
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Challenges with the ACES program. When asked in the survey about challenges, some 
coaches cited the difficulty of meeting and communicating with students as needed in = 5), lack 
of resources ( n = 5), and getting school support while building a new program in = 3). Categories 
of findings and examples of responses may be seen in Appendix A, Table A5. Seven RCs left a 
response about challenges, but no more than two responses with similar challenges were reported 
(Appendix A, Table A6). Some of the challenges that emerged in an earlier interview of coaches 
and RCs were student attendance in activities (9 of 20), communication about the ACES 
program (8 of 20), implementing a program while it’s still being developed (4 of 20), recruiting 
students (4 of 20), and the office location for coaches (4 of 20). 

Suggestions for the ACES program. As shown in Appendix A, Table A7, one half of the 
coaches made suggestions in the survey surrounding resources needed to improve the program 
such as office space, buses, computers, and classrooms. This coincides with an earlier interview 
of coaches and RCs where one fifth of the interviewees suggested different office spaces for the 
coaches. Also in the survey, several made suggestions to increase the communication between 
the coaches and school staff. In an earlier interview with coaches and RCs, just over half of the 
interviewees suggested frequent and clear communication from leadership and between coaches 
and counselors. Other suggestions by one or two interviewees were: wait to expand the program 
next year, more communication about the program with students, involve those at the school in 
decision making, make ACES into a course, have a central place for record keeping, and have 
more after-school dates available. 

Of the eight RCs who provided suggestions, one half were related to improving communication 
between all parties (Appendix A, Table A8). A variety of other suggestions were given, but due 
to lack of frequency, no common themes emerged. Examples of suggestions included: change 
the application deadline, involve school staff members more, provide coaches with access to 
student records, offer more flexible recruitment process, etc. 

Evaluation Question: To what extent did the ACES program increase student knowledge 
and assist in their participation of the college application and financial aid process? 

Data on college applications and acceptances, financial aid applications, and scholarship 
applications among all graduating ACES students will be reported in a subsequent ACES year 
one outcome report. This section presents student and staff perceptions, obtained through 
surveys, on student degree expectations, the effect of application fees, counselors’ approach to 
college planning, and ACES program assistance with the college process. 

Student Responses 

College degree expectations. When students were asked what would be the highest 
college degree they expected to receive, 28% of the student respondents said a bachelor’s degree, 
and another 56% reported a master’s or doctor’s degree (Table 16). 


Program Evaluation 


CONFIDENTIAL 


18 


DRAFT 


Evaluation of ACES Year One 



Montgomery County Public Schools 


Office of Shared Accountability 


Table 16 

Student Survey: Highest Degree Expected to Receive among Respondents 

A =458 


n % 


High School Diploma 

27 

5.9 

Associate’s or trade/business/technical degree 

21 

4.6 

Bachelor’s degree 

130 

28.4 

Master’s degree 

139 

30.3 

Doctorate (Ph.D.) or professional degree 

116 

25.3 

Undecided 

25 

5.5 


College application fees. Almost two thirds (65%) of seniors surveyed who applied to a 
4-year college reported that they were able to apply to the number of colleges they wanted; 
whereas more than one third (35%) reported that they were not able to apply to all the colleges 
they wanted because of the application fees. 

ACES program assistance. As shown in Table 17, almost all students surveyed 
(90-94%) agreed or strongly agreed that the ACES program assisted them with exploring 
colleges and universities (50% strongly agreed), assisted them with exploring majors and career 
interests (47% strongly agreed), helped them prepare academically for college applications (45% 
strongly agreed), and provided their parent or guardian with helpful information (47% strongly 
agreed). 


Table 17 


Student Survey: Experience with Program Assistance 



Strongly 

Agree 

Agree 

Disagree 

Strongly 

Disagree 


n 

% 

n 

% 

n 

% 

n % 

The ACES program assisted me with exploring 








colleges and universities to which I might want to 
apply. ( N = 446) 

223 

50.0 

194 

43.5 

24 

5.4 

5 1.1 

The ACES program assisted me with exploring college 
majors and career interests. ( N = 452) 

214 

47.3 

207 

45.8 

27 

6.0 

4 0.8 

The ACES program helped prepare me academically 
for the college application process. ( N = 443) 

199 

44.9 

212 

47.9 

26 

5.9 

6 1.4 

The ACES coach provided my parent(s)/guardian with 
helpful information and guidance. ( N = 445) 

208 

46.7 

194 

43.6 

37 

8.3 

6 1.3 


The Grade 12 ACES students were asked to rate a series of statements about program assistance 
(Table 18). Almost all students (a range of 97-99%) agreed or strongly agreed that they were 
able to get the guidance they needed for the college application process (63% strongly agreed), 
the program assisted them in understanding the college application process (65% strongly 
agreed), the program assisted them with how to apply for financial aid (67% strongly agreed), 
and the program assisted them in understanding how to apply for scholarships (63% strongly 
agreed). 
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Table 18 


Student Survey: Experience with Program Assistance - Grade 12 Only 



Strongly 

Agree 

Agree 

Disagree 

Strongly 

Disagree 


n 

% 

n 

% 

n 

% 

n % 

Overall, I was able to get the guidance I 
needed for the college application process. 
(N = 259 ) 

162 

62.5 

94 

36.3 

3 

1.2 

0 0.0 

The ACES program assisted me in 
understanding the college application 
process. ( N = 260) 

169 

65.0 

86 

33.1 

5 

1.9 

0 0.0 

The ACES program assisted me in 
understanding how to apply for financial 
aid. (N = 259) 

173 

66.8 

81 

31.3 

5 

1.9 

0 0.0 

The ACES program assisted me in 
understanding how to apply for 
scholarships. (N = 261) 

164 

62.8 

89 

34.1 

8 

3.1 

0 0.0 


Grade 12 respondents who were accepted into a 4-year college were asked the helpfulness of the 
ACES program to their acceptance. Most respondents (94%) found the program very helpful 
(53%) or helpful (41%) to their acceptance into a 4-year college. 


Staff Responses 

College plans. RCs reported that the counselors, CCIC, or both work with students on 
their college plan, with no typical method reported across all the schools (Table 19). 

Table 19 


Resource Counselor Survey: Approaches to Handling Student College Plans 
(A = 10) 


College Plan Approaches 

n 

A student’s counselor typically works with the student on a college plan. 

3 

The CCIC typically works with the student on a college plan. 

1 

Both CCIC and student’s counselor above were chosen. 

4 

A CCIC will start a college plan with a student and then a counselor will 
finish/follow up. 

2 


ACES assistance with college. As shown in Table 20, almost all coaches strongly agreed 
that ACES motivates students to go to college who otherwise might not have gone (n = 9); that 
ACES serves students who are motivated to go to college, but face barriers (n = 8); and that 
ACES helps students get accepted into a 4-year college (n = 8). A few RCs disagreed that ACES 
motivates students to go to college who might not otherwise (n = 3) and that ACES helps 
students get accepted into a 4-year college ( n = 3). 
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Table 20 


ACES Coach and RC Survey: Respondents’ Perception on ACES Program 


Resource Counselors (N = 10) 

ACES Coaches (N = 10) 

Strongly 

agree Agree Disagree 

Strongly 

agree Agree Disagree 

n n n 

n n n 

ACES motivates students to go to college 5 2 3 

who otherwise might not have gone. 

9 1 0 

ACES serves students who are motivated 6 3 1 

to go to college, but face barriers. 

8 2 0 

ACES helps students get accepted into a 4 2 3 

4-year college. a 

8 2 0 

ACES helps students achieve financial aid , . 

for college." 1 

7 3 0 


■"Resource counselors, n = 9. 


Summary 

Students were recruited to the ACES program in a variety of ways across the schools. Students 
learned about the program directly through the ACES coach or their counselor, a presentation 
during class, an assembly, or school advertisement. Although students met one of the qualifying 
criteria to be accepted into the program, schools focused on varying criteria and used a variety of 
methods to identify students to recruit. 

ACES students who completed an end-of-year survey (49% of all ACES students) had very 
positive ratings about their experience and the overall quality of the ACES program. They 
reported the coach and program were helpful in guiding them through college information and 
the application process, and they would recommend the program to other students. The aspect of 
the program they especially liked was the coach. Almost all students who left a comment at the 
end of the survey left a positive comment about the program or a thank you to the program or 
their coach. Some students expressed challenges with the workshop schedule or scheduling 
meetings with the coach and would like to see more options or flexibility in this area. Also, 
some students were not clear as to whom in the counseling office to see for assistance. Students’ 
suggestions on improving the ACES program were to offer more convenient meeting times, 
more field trips, more scholarship information, and more individualized attention. One student 
also pointed out a possible misconception that some students believed they had to attend 
Montgomery College if in the program. 

All 20 ACES coaches and RCs (100% response rate) responded to an end-of-year survey and 
participated in midyear interviews. Most thought the collaboration between the coach and RC 
was successful, and most RCs strongly agreed that the coach was able to provide additional 
support to students in the program. Most coaches strongly agreed that ACES motivated students 
to go to college who might not otherwise have gone (although a few RCs disagreed); ACES 
helped students get accepted into a 4-year college (although a few RCs disagreed); and overall, 
ACES has met the needs of students. 
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Although most ACES coaches and RCs reported working together on various aspects of ACES, 
such as exchanging information about ACES activities and students, there is variation among the 
respondents on what has been implemented well and what presented challenges. Challenges that 
were cited by at least some coaches and RCs were the process for recruiting and enrolling 
students, collaboration with the CCIC, and the coach’s access to student data. At least half of the 
coaches indicated a need for cooperation between them and the school staff members; the ability 
to meet with all students; and resources such as office space, computers and the use of 
equipment. When asked, 7 of 10 coaches reported that the student caseload was too high. Some 
RCs indicated that they are not always kept informed about the implementation of the program 
and would like to see more communication between all parties. 

It is pertinent to point out that some of the main findings from the midyear interviews of ACES 
coaches and RCs centered on more frequent communication to school staff members about the 
program and its expectations as well as clarifying roles among the ACES coach and counseling 
staff. The findings from the survey administered in May did not highlight these concerns, 
indicating a shift in improvement in these areas. In fact, the program staff organized a full-day 
retreat for the summer of 2014, and a follow-up retreat in October, to bring ACES coaches and 
MCPS staff members directly involved with the program together to focus on clarification of 
roles and responsibilities, team building, and sharing of best practices. 


Recommendations 

During this first year of implementation, ACES staff, ACES coaches, and MCPS school staff had 
to establish procedures and roles for the program, create a comprehensive database for capturing 
student information, locate office space for the coach and meeting space for ACES activities, 
coordinate the location and schedule of student meetings, complete the recruitment of students, 
etc. Despite these first year challenges, there were many successes. Surveyed students were very 
happy with the program and its guidance for college applications; staff also agreed that the 
program was successful. 

The following recommendations are to provide feedback for the program’s improvement and 
ongoing development and are based on findings from this part of the evaluation. 

• Continue to establish helpful and positive relationships between coaches and students. 

• Continue to build relationships between the ACES coach and school staff. 

o Areas of concentration, depending on specific school, include: procedures for sharing 
student information, coach’s use of equipment, attendance at counseling department 
meetings, clarification of roles for staff and for students, and regular communication 
between counseling staff and coaches. 

• Continue to provide clear and frequent communication with school staff members and 
ACES coaches about program updates and expectations. 

• Explore providing more resources for coaches such as classroom and/or office space 
availability and equipment. 

• Explore scheduling of ACES activities to accommodate more students, 
o Consider duplicating important workshops and information sessions. 
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o Consider altering coach’s schedules to more closely align to MCPS school schedules 
and increase coach’s availability (i.e., Fridays at school, spring breaks). 

• Explore possible misconceptions among some students and some counseling staff that 
students are required or encouraged only to attend MC and will not receive help applying 
to a 4-year college. 

• Consider adding more college field trips, especially early in the school year, and 
information regarding scholarship opportunities. 

• Schedule time for ACES school-level teams from the 10 schools to come together to 
clarify roles and responsibilities and share best practices with each other. 
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Appendix A 

Table A1 

Student Survey: Suggestions for ACES (Open-ended) 

N= 152 

n % 

Workshop + Activity Suggestions 

More college trips/more field trips 26 17.1 

More scholarship opportunities/information 13 8.6 

Offer other topics (e.g. college tips, personal finance, careers, world after high school, etc.) 1 1 7.2 

More on choosing a college/college topics 10 6.6 

More workshop opportunities 7 4.6 

More SSL opportunities 6 3.9 

Coach’s Approach 

More individualized/more one-on-one 15 9.9 

More motivation/involvement 13 8.6 

Make time for everyone who needs it/treat everyone the same 9 5.9 

More frequent communication/more emails/better organization such as a calendar 9 5.9 

Meet with other students/group meetings + activities/students help students 6 3.9 

Push 4 year colleges more/don’t just target MC and USG 3 2.0 

Scheduling 

Meeting times (i.e., more convenient, more flexible, more times, more times during the 
school day, etc.) 22 14.5 

More available workshop times/repeat 5 3.3 

Meet more often 5 3.3 

Start earlier in High School career/college visits in junior year 5 3.3 

Meet at lunch/during school hours 4 2.6 

Change SAT prep time/offer SAT prep after school 3 2.0 

Coach Support 

More coaches/get an assistant coach/get coaches more help 5 3.3 

Offer more food 5 3.3 

More/better computers 4 2.6 

More space to meet with students 3 2.0 

Participation 

Advertise ACES program more 2 1.3 

Students should be committed/have obligation to participate in program 2 1.3 

Other 

Other suggestions 15 9.9 

Note. More than one comment could be given. 
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Table A2 

ACES Coach Survey: Support Needed (Open-ended) 
(A = 10) 

Needed Supports n 

Cooperation between ACES & the school/school staff 5 

Schoolwide awareness of the ACES program. 

It would also help if ACES recruitment was embraced as a schoolwide 
initiative during which all key stakeholders actively participated. 

Supportive collaboration with the CCIC. 

Flexibility in meeting students in the last 5 to 10 minutes of an elective class 
once a week. 

To be considered a part of the school. 

Need for resources (office, classroom, computers, etc.) 5 

Space is critical to provide SAT prep for the ACES students in the upcoming 
school year. 

It would be nice to get an office with a few computers in order to work with a 
small group of students at a time. 

I would also like to be able to have a permanent space/time to offer student 
workshops. 

Make school resources more available. 

The lack of resources has hindered the program. 

Databases ( ACES & MCPS) 3 

Access to certain databases that would give me more updated information 
regarding student’s academic status (grades). 

Access to student grades and progress reports. 

A functioning and completed ACES database for all of our ACES data. 
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Table A3 

ACES Coach Survey: Positive Aspects (Open End) ( N = 10) 


Categories 

Examples 

n 


Collaboration with (school) staff. 



The support from my school is appreciated. My resource counselor 
and I work well together. 


School and staff support + 
communication 

The school principal and resource counselor have been helpful. 

Being adaptive as changes take place help in an environment where 
things are constantly changing and where we as coaches have no real 
authority in the school system even though we are all high 
performing professionals. 

Collaboration with counseling department -autonomy given to me by 

administration - a good office space - a very good relationship 

with CCIC... 

6 


The collaboration from the Counseling Department has been 
phenomenal. It really does take a village to make the students 
become aware that they can be successful. 


Individual meetings with 
students and/or parents 

Individual meetings with students and parents (especially with 
financial aid process). 

I think that the students have been able to benefit greatly from the 
services and individual assistance. The parents also appreciate the 
one-on-one support as well. 

5 

Coaches time flexibility to 
meet with parents and 
provide multiple 
workshops 

The flexible schedules of the coaches allowed parents who work 
multiple jobs to have better access to school resources. 

Providing workshops at multiple times for students. 

2 

Other 

Providing snacks 

Successful kick-off 

Space to meet and conduct workshops 

3 
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Table A4 

Resource Counselor Survey: Positive Aspects (Open End) ( N = 7) 


Categories 

Examples 

n 

Collaboration and communication 
from ACES coach 

My ACES coach is wonderful. She collaborates with 
the counseling department on a regular basis and 
makes sure that we are kept informed. 

4 


The coach was very proactive in working with 
counselors and staff. 


Support and interaction provided 
by ACES coach to students 

I can see that the students in the ACES program have 
clear goals. Before ACES, some of the students 
seemed to be clueless about the college preparation 
process. 

The students were motivated to pursue a college track 
and received the extra support they needed. 

4 


The coach was very proactive in reaching out to 
students. 


Other 

100% of the students applied to college. 

1 
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Table A5 

ACES Coach Survey: Challenges (Open End) (A = 10) 
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Table A6 

ACES Resource Counselor Survey: Challenges (Open End) ( N = 7) 


Categories 

Examples 

n 

To include 12 th graders in first 
year 

Having the coach spend the first semester getting to 
know the seniors and miss out on building a stronger 
relationship with the juniors. The coach should have 
only worked with juniors this year, so next year they 
would already know the rising seniors well (their 
needs and their plans for college). 

2 


Meeting with every student in ACES . 



Understanding the logistics of the program and what 
role does the school play when it comes to preparing 
our students for the ACES program. 


Division of responsibility 

Creating clear responsibilities and domains for the 
ACES Coach, Counselors, and CCIC as it relates to 
working with students we all share. 

2 

Recruitment and Referrals 

Recruiting students was difficult at first and is still an 
intensive process. 

2 

Other 

Coach’s access to student information. 
Availability of coaches. 

Family involvement. 

Workshops not held. 

4 
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Table A7 

ACES Coach Survey: Suggestions (Open End) (N = 10) 


Categories 

Examples 

n 


Provide a permanent space for ACES workshops and 
meetings. Additional options for afterschool 
transportation so that more afterschool workshops can 
be held. 


Provide resources 
(e.g., space) 

Having a classroom for each ACES coach would 
prove a tremendous asset. It would eliminate having to 
chase down a space and utilize our time appropriately 
in implementing the program. . . Even a smaller size 
classroom would be beneficial. . .. 

5 


The only challenge has been having an ongoing space 
to provide workshops. 


Increase communication 
between coaches and 
counseling department/ 
school staff 

I suggest the ACES coach and counseling staff meet 
on a regular basis regarding the program and students. 

Strengthen the collaboration with school-based MCPS 
staff. 

4 


Encouraging schoolwide involvement dining 
recruitment. 


Recruitment and selection 
of students 

I think the schools should be more active and aware of 
what ACES is doing on a broader level. 

3 

Other 

Minimize caseload. 

1 
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Table A8 


Resource Counselor Survey: Suggestions (Open End) ( N = 8) 


Categories 

Examples 

n 


Communication between the ACES Coach and RC is 
imperative in order for me to address questions on this 
survey. There is a chance the ACES Coach has 
surpassed every expectation, however without my 
knowing, I am unable to express her work. 


Communication between all 
parties 

Communication between all stakeholders with 
advance notice of changes would be helpful. 

A group meeting with all ACES readiness staff and 
ACES staff to help coordinate services. 

4 


Efforts should be made to model and set a tone that 
this is a team and everyone is expected to work 
together. This division plays out with issues with 
ACCUPLACER, enrollment with college classes and 

recruitment for MC all be expected to team 

together so there is less confusion about a range of 
processes and issues. 


Other 

Change application deadline. 

Involve school staff more. 

Provide coaches access to student records. 
More flexible recruitment. 

More availability among coaches to work with 
students. 

1 
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Appendix B 


Attachm cnt A 
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